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Abstract 

In this paper, two new classes of lower bounds on the probability of error for m-ary hypothesis testing are proposed. 
Computation of the minimum probability of error which is attained by the maximum a-posteriori probability (MAP) 
criterion, is usually not tractable. The new classes are derived using Holder's inequality and reverse Holder's inequality. 



JC The bounds in these classes provide good prediction of the minimum probability of error in multiple hypothesis testing. 
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qq , The new classes generalize and extend existing bounds and their relation to some existing upper bounds is presented. It 
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\y-\ , is shown that the tightest bounds in these classes asymptotically coincide with the optimum probability of error provided 
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^^ , by the MAP criterion for binary or multiple hypothesis testing problem. These bounds are compared with other existing 
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>• lower bounds in several typical detection and classification problems in terms of tightness and computational complexity. 
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I. Introduction 

Lower bounds on the probability of error are of great importance in system design and performance analysis 
in many applications, such as signal detection, communications [T], classification, and pattern recognition [2]. 
It is well known that the minimum probability of error is attained by the maximum a-posteriori probability 



(MAP) criterion, however, its probability of error is often difficult to calculate and usually not tractable. 
In such cases, lower bounds on the probability of error are useful for performance analysis, feasibility study 
and system design. These bounds can be useful also for derivation of analytical expressions for the family of 
Ziv-Zakai lower bounds (ZZLB) for parameter estimation [3]. One of the difficulties in computation of the 
ZZLB is that they involve an expression for the minimum probability of error of a binary hypothesis problem. 
Analytic expressions for lower bounds on the probability of error may be useful to simplify the calculation 
of the bound. Another application of these bounds is a sphere packing lower bound on probability of error 
under MAP of the ensemble of random codes [3]. 

Several lower bounds on the probability of error have been presented in the literature, for specific problems, 
such as signals in white Gaussian noise [5], [6], and for general statistical models. The general bounds can 
be divided into bounds for binary hypothesis problems [TUllj and bounds for multiple-hypothesis problems 
[4], |12W21j . Several lower and upper bounds utilize distance measures between statistical distributions, like 
Bhattacharyya distance [TJ, [8], Chernoff [9J, Bayesian distance [17] , Matusita distance |19] , and the general 
mean distance [20] . j21| . Two classical lower bounds on the multiple- hypothesis error probability that have 
been used in proving coding theorems are the Shannon [12] and Fano [13] inequalities. The relations between 
entropy and error probability have been used to derive the bounds in [UQ3MT5]. The bound in [16] has 
been derived using proofs of converse channel coding theorems in information theory. In addition, there are 
several ad-hoc binary hypothesis testing bounds that directly bound the minimum function on the a-posteriori 
probabilities. This class includes the "Gaussian-Sinusoidal" upper and lower bounds [TT] and the exponential 
bound [ID], that are found to be useful in some specific cases. A brief review of some existing lower bounds 
on the probability of error is presented in Appendix B and in |22| . 

Practical and useful lower bounds on the probability of error are expected to be computationally simple, 
tight, and appropriate for general multi-hypothesis problems. In this paper, two new classes of lower bounds 
with the aforementioned desired properties are derived using Holder's inequality and reverse Holder's inequal- 
ity. The bounds in these classes provide good prediction of the minimum probability of error in multiple 
hypothesis testing and are often easier to evaluate than the MAP probability of error. These bounds are 



compared with other existing lower bounds. In addition, it is shown that the new classes generalize some 
existing lower bounds |3], [T7], [2T1I23 . 24j . The tightest lower bound under each class of bounds is derived and 
it is shown that the tightest bound asymptotically coincides with the optimum probability of error provided 
by the MAP criterion. 

The paper is organized as follows. The new classes of bounds are derived in Section [II] and the bounds 
properties are presented in Section IIHI In Section IIVI simple versions of the ZZLB for parameter estimation 
are derived using the proposed classes of bounds. The performances of the proposed bounds for various 
examples is evaluated in Section [V] Finally, our conclusions appear in Section IVII 

II. General classes of bounds on probability of error 

A. Problem statement 

Consider an M-ary hypothesis testing problem, in which the hypotheses are Oi, i = 1, . . . ,M with the 
corresponding a-priori probabilities P(0i), i = 1, . . . , M. Let P(#j|x), i = 1, . . . , M denote the conditional 
probability of Oi given the random observation vector, x. The probability of error of the decision problem is 
denoted by P e . It is well known that the minimum probability of error obtained by the MAP criterion, is 
given by [H] 



p(min) _ y _ JTJ 



max PiOAx) 
=1,...,M 



(1) 



where the MAP detector is 



Omap = arg max P(0\x) . 
ee{8 u ...,e M } 

However, the minimum probability of error in (prj is often difficult to calculate and usually not tractable. 

Therefore, computable and tight lower and upper bounds on the probability of error are useful for performance 

analysis and system design. 

B. Derivation of the general classes of bounds 

Consider the above M-ary hypothesis testing problem with detector = #(x). The detector 0(x) is as- 
sumed to be an arbitrary detector that decides on one of the hypotheses with positive (non-zero) a-posteriori 



probabilities. That is, in the case where P{9j\x) = we assume that 0(x) ^ 9j with probability 1 (w.p.l). 

Let 

1 if 0/0 



A 



u(x, 9) = 1 §¥=9 = { 
where 6 is the true hypothesis. It can be verified that 



(2) 



if 9 = 6 



P e = E[n(x,0)]=E[|n(x,0)n 



(3) 



and 



P e = 1 - E [1 - u(x, 6)} = 1 - E [|1 - u(x, 6)\ p ] 



(4) 



for every p > 0, where P e is the probability of error of the detector 0. Then, according to Holder's inequality 
and reverse Holder's inequality [22], [25] : 

(5) 



p-1 r 



Ep[\u(x,6)\ p ]E— \vi(x,0)\*=i >E[|«(x,0)ui(x,0)|], p>l 



and 



E[|(l-u(x,0))t> 2 (x,0)|] >E P |l-n(x,0)|p E 1 ^ |v 2 (x,0)|^ 



p> 1 



(6) 



for arbitrary scalar functions Ui(x, 0) and u 2 (x, 0). 

By substituting of © and (jl]) into §5§ and ©, respectively, one obtains the following lower bounds on the 
probability of error: 



P e >E p [|n(x,0)« 1 (x,0)|]E 1 - p Mx,0)|^ 



, p>\ 



(7) 



p-i 



P e > 1 - EF [| (1 - u(x, 0)) «2(x, 0)|]E— |«a(x, 0)| * 



p > 1 . 



(8) 



By substituting different functions fi(x, 0), t> 2 (x, 9) in ([T])-©, one obtains different lower bounds on the 
probability of error. In general, this bound is a function of the detector via tt(x, 0). The following theorem 
states the condition to obtain valid bounds which are independent of the estimator. 



Theorem 1: Under the assumption that P(0j|x) > Vx £ \ and 9i, i = 1, ..., M, a necessary and sufficient 
condition to obtain a valid bound on the probability of error which is independent of the detector 9, is that 
the functions v\ (x, 9) and v<i (x, 9) should be structured as follows 



Vk(*,0i) 



Cfc(x) 



fc = l,2, i = l,...,M 



(9) 



where Ci(') an d C2(") are arbitrary functions of the observations x and with no loss of generality they should 
be chosen to be non-negative. 
Proof: In Appendix A. 

Using ([9]) it is shown in Appendix A that using {ffc( x >$i)}fc=i,2 defined in Q 



E[Kx ) ^ 1 (x,^)|] = (M-l)E[Ci(x)], 



(10) 



E[|(l-«(x,0))«2(x,0)|]=E[C 2 (x)], 



;n) 



and 



E 



\vi (x, 9)\p- 



E 



' _JL_ M 1 



i=i 



(12) 



E 



|u 2 (x,0)|i-*> 



E 



M 



cr p (x)^p^(^ix) 



i=l 



(13) 



By substitution of (fT0|) . (fl~2]l into (J7|), and (fTT|) . ([13]) into (jSJ), the new classes of lower bounds can be rewritten 



as: 



P e >(M-l)^[Ci(x)]E 1 ^ 



j_ M i 

crw^rp— (^ix) 



i=l 



p > 1 



(14) 



p-i 



P e > 1 - Ep [c 2 (x)]e- 



Af 



c 2 i - p (x)E p ^(^i x ) 



i=\ 



, p> 1 . 



(15) 



C. The tightest subclasses in the proposed classes of lower bounds 



According to Holder's inequality [25] 



EPKiMlE 1 -" 



' j_ M i 



4=1 



and it becomes an equality iff 



> E 



E 



11 



( -JL. M i \ 

cf(x) cr(x)^p^^ix) 

ff^P^i(^|x)j 



Ci(x) = C1 cr 1 (x)^p T ^(^ix) 



where c\ denotes a constant independent of x and 9i, i = 1, . . . , M. In similar, 



Ep [( 2 (x)]eV 



ili 



cr p (x)^p^(^ix) 



4 = 1 



and it becomes an equality iff 



> E 



E 



iii 



i\i 



8=1 



t M 

^PF^(^|x) 



£-1 
P 



V«=l 



c 2 (x) = C2 cr p (x)E p?ZT ^i x ) 



4 = 1 



1-p- 



P-l 

P 



cf(x) c 2 1 - p (x)E Pi ^^i x ) 



(16) 



(17) 



(18) 



(19) 



where c 2 denotes a constant independent of x and 9i, i = 1, . . . , M. Thus, the tightest subclasses of bounds 

in the two classes are: 

l-p-i 



Pe > Bjp = (M - 1) P E 



' M \ 

^M^|x) 



, Vp> 1 



(20) 



P e >i?f =1-E 



' M 

^Pp^(^|x) 



p-l 

p 



a=l 



Vp> 1 



(21) 



obtained by substituting (fPTI) and ([19]) in (fllj) and (fl~5l) . respectively 



-D. Simplifications of the bound 



M 



The bounds in (|20p and ()2ip can be simplified using Jensen's inequality |26j . Let £i = ^ P 1 -p (0j|x) > 0, 

4 = 1 

than for p > 1 B p = e 1 ~ p is a convex function of £\ > 0. According to Jensen's inequality for convex 



functions 



flM = (M- 1) P E 



' M \ 



1-p- 



?M 



> j^W = (M - 1) P E 



i-p 



' M 



«=i 



Vp > 1 . (22) 



.(2) ^ - - - - M _ 



In similar, B p = e 2 p is a concave function of the positive term £2 = X] P^ 1 (#i| x )- According to Jensen's 

*=1 



inequality for concave functions 

' Af 



p£ 2) = 1 - E 



E P ^ 



p 



X 



vi=l 



> J^ 2 ) £ 1 - E^ 



jU 



^P— (^|x) 



Vjj>1. 



III. Properties of the proposed classes of bounds 



A. Asymptotic properties 

According to [25] (Theorem 19, page 28), for any sequence of nonnegative numbers, ai, . . . , om 

/ M \ s / M \ t 



E«M ME 



oj , VO < s < t 



\i=l / \i=l 

and thus, the term within the expectation in (|20p 

/ M \ 1 -p 



^Pi^(^|x) 



1 



a=l 



A7 

2=1 



1 \ p- 1 



(23) 



(24) 



is a decreasing function of p for all p > 1. Therefore, in the binary case, the bound in (I20p satisfies 



PW>PW, Vl<p<r, M = 2. 



(25) 



In particular, for p — > 1 + , the bound in (|20p becomes 



BQ = lim Pi 1 ) = E 



'1+ 



p-*-!" 1 



min P (OAx.) 
i=l,2 



1-E 



maxP (0f|xl 

i=l,2 



(26) 



which is the tightest lower bound on the probability of error in the first proposed class of lower bounds for 
M = 2. Thus, for the binary hypothesis testing the bound in (|20|) with p — > 1 + is tight and attains the 



minimum probability of error, presented in ([I]). 

M 



»=l 



p-i 



In similar, using (124p the term I Y^ Pp- 1 (#j|x) J from (|2ip . which is the -^j norm of {P(#i|x)} i=1 



.Af' 



is a decreasing function of p for all p > 1. Therefore, in the general case 



B^>B^\ Vl<p<r, VM 



(27) 



In particular, for p — > 1 , the bound in (|2ip becomes 



Bg = lira B^ = 1 - E 



max P(0,-|x) 

i=l,..,M V ' ^ 



(28) 



which is the minimum probability of error, obtained by the MAP criterion. Thus, for the M-hypothesis testing, 
the bound in ()2ip with p — > 1 + is tight and attains the minimum probability of error presented in ([I]). 

B. Generalization of existing bounds 

In this section, we show that the proposed classes of bounds generalize some existing bounds. In particular, 
the lower bounds in @], [T7], and [21] can be interpreted as special cases of the proposed general M-hypotheses 
bounds, presented in (|20p and (|2ip . In the binary hypothesis testing, the bound in (|20p with p = 2 can be 
written by the following simple version: 



P e >E[P(0i|x)P(0 2 |x)] 



(29) 



which is identical to the harmonic lower bound [JJ and to the Vajdas quadratic entropy bound [23], [23] with 
M = 2. 

In the multiple hypothesis testing the bound in (|2ip with p = 2 can be written in the following simple 
version: 



P, > 1 - E 



M 






(30) 



which is identical to the Bayesian lower bound [3], [T7], B^ Bayes3 > , described in Appendix B. The bound JBp 
in fl23D with p = 2, M = 2 is 



P e > JBf ] = 1 



N 



E 



M 



E p2 (^i x ) 



i=l 



(31) 



is identical to the Bayesian lower bound [T7], B^ Bayes2 \ described in Appendix B. In addition, in the multiple 
hypothesis testing, the subclass of bounds in (|2ip is a "general mean distance" class of bounds presented in 
Appendix B in (|62p . 



C. Relation to upper bounds on minimum probability of error 

In [3], a class of upper bounds on the MAP probability of error for binary hypothesis testing is derived 
using the negative power mean inequalities: 



p(min) < 2<J>-1)e 



^P(0i|x)^F 



1-p 



J=l 



(32) 



for any p > 1. It can be seen that this class of upper bounds is proportional to the proposed tightest subclass of 
lower bound in (120j) with a factor of 2 P ~ 1 . This factor controls the tightness between upper and lower bounds 
in the probability of error for binary hypothesis testing. This upper bound coincides with the proposed lower 
bound Bp in the limit of p — > 1 + . 

In [21], the "general mean distance" is used in order to derive upper bounds on the MAP probability of 
error. One subclass of upper bounds presented in this reference is 



p{min) < g{GMD3) _ ^ _ M^E 
e p 



p-l 

M \ — 



5^pA(^| X ) 



,i=l 



p > 1 . (33) 



It can be seen that 1 — B p = M p ( 1 — B p I . Thus, the computations of ([33]) for specific hypothesis 



testing problems can be utilized to compute the lower bounds in (|20j) for the same problems. 

IV. Application: simple versions of the Ziv-Zakai lower bound 

The new classes of lower bounds on the probability of error can be used to derive simple closed forms 
of the ZZLB for Bayesian parameter estimation. A critical factor in implementing the extended ZZLB is 
the evaluation of the probability of error in a binary detection problem. The bounds are useful only if the 
probability of error is known or can be tightly lower bounded. Thus, the new classes of lower bounds in (|14p . 
(|15p can be used in order to derive lower bounds on the ZZLB, providing less tighter MSE bounds which may 
be easier to compute. Note that the derivation in this section is performed under the assumption that x is 
continuous random variable. Extension to any random variable x with E[x 2 ] < oo is straightforward. 

Consider the estimation of a continuous scalar random variable 4> £ $, with a-priori probability density 
function (pdf) /</,(</>), based on an observation vector x € x- The pdf's /^,| x (-|x) denotes the conditional pdf 



10 



of (f) given x. For any estimator </>(x) with estimation error e = 0(x) — eft, the mean-square-error (MSE) is 

2" 
defined as E 



x 



. The extended ZZLB is [27] 



oo / oo 



E 



1 



> ZZLB = - I 

o 



(/</>(¥>) + /</>(<£ + h)) P m in(p, p + h)dp> 



> hdh 



where P m i n (p,p + h) is the minimum probability of error for the following detection problem: 

#0 : /x|H ( x ) = /x|</,0<#) 
Hi : / x |Hi(x) = / x |^(x|v + M 



with prior probabilities 



H#n 



/*(¥>) 



-, P^) = 1 - P(P 



AM + U(f + h) 
The operator V returns a nonincreasing function by filling in any valleys in the input function 



(34) 



(35) 



(36) 



Vf(h) = maxf(h + 0, he 



(37) 



Since /,*(<£>), f<j>(p + h), and P m in(p,<P + h) are non-negative terms, the inner integral term in (|34p can be 
lower bounded by bounding P m in(p, <p + h). Thus, 



E 



(x) 



oo f oo 

> ZZLB >C P = - IV 





(/*(¥>) + U& + h )) LB (^ h)dip}h dh 



(38) 



where LB(p,h) is any lower bound on the minimum probability of error of the detection problem stated in 
(j35j) . By substituting the lower bound on the probability of outage error from (fT4"j) and (fl~5j). respectively, 
in (|38p with M = 2 and using arbitrary non-negative functions (i( x )> C2( x ) one obtains different MSE lower 
bounds. In particular, by substituting LB = B p and LB = B p from ([20]) and ([2T]) . respectively in (|38|) . 
one obtains the tightest classes of MSE lower bounds 



E 



oo / oo 

v [e 



and 



E 



x) 

klAl 







o 



f£ P (v\*) + f£ P (<P + h\^ 



dtp 



}hdh, \/p>l (39) 



E 



V&w+fi 1 



/rM x )+/r(n%) 



' oo 



E 



U\* teW + /^|x (¥> + /*W - /I,; 1 Mx) + /r 1 ( v + &|x) 






£-1 
P 



dip > hdh 



dp 



> hdh, 



(40) 
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\/p > 1. For p — > 1 + , the bounds in (|39|) and (I40J1 become 



E 



0(x) - 



oo ( oo 
2 



>^Jvl J E [min (/^, x (p|x) , / , x (p + fc|x))] d^ 



► fcd/i, (41) 



o 
which coincides with the ZZLB as presented in [28]. 

V. Examples 

A. Bounds comparison 

Fig. Q] depicts the lower bounds B p and i?p , presented in (|20p and (|2ip . for the binary hypothesis problem 

against the conditional probability P(9\\x), for different values of the parameter p and given x. It can be seen 

that the bounds in (j20|) and (|2ip become tighter as p decreases and that for given p, B p is always tighter 
thanB^. 

/2 s ) *7-m, 

Fig. [2] depicts the lower bound B p , presented in (|2ip . for the binary hypothesis problem against the 
conditional probability P(#i|x), for different values of the parameter p and given x. The new bound is 
compared to the bounds B{ Gauss ~ sm ) anc ; b( atlb ) with a = 5, presented in Appendix B. It can be seen that 

(2) 

Bp becomes tighter as p decreases, and that for p = 1.1, the new bound is tighter than the other lower 
bounds almost everywhere. 

B. Example: Binary hypothesis problem 

Consider the following binary hypothesis testing problem: 

e 1 : /(x|0i) = \ie~ Xlx u(x) 



(42) 



72 



: f(x\9 2 ) = X 2 e- X2X u(x) 



where u(-) denotes the unit step function, P{6\) = P(&2) = o> an< ^ Ai = ^-^ > Ai. For this problem, the 
bounds in (|20p with p = 2 and p = 1.5 are 

(i) _ 1 j-, ( A2 1 Ai — 2A 2 _ A2 

1 



2 V Ai — A2 Ai — A2 A 



R (i) _ 1 p f A 2 - 1 .-. A 2 _ Aj 

^■ 5 2 2 X V 2(Ai-A 2 )'2' 2(Ai-A 2 )' A? 
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,(min) 



_ _B 11 ' B*'' 

1.1 1.25 



?(D 



3 (2) 

1.1 



-+- B 



(2) 
1.25 



3 (2) 
1.5 




0.4 0.6 

P(e 1 |x) 

Fig. 1 

The proposed LOWER BOUNDS, Bp^' AND Bp" J WITH p = 2, 1.25, 1.1 AS a function of the conditional 

PROBABILITY P(#i |x) FOR BINARY HYPOTHESIS TESTING. 



and the bounds in (1211) are 



B 



(2) 
i 

9-1 



1 - - 2*1 

2 



Ai 



Ai 



A 2 



c(Ai-A 2 )' q g , (Ai-A 2 )' A f { 



Vg = 2,3,... 



where 2*1 is the hyper geometric function [29]. Several bounds on the probability of error and the minimum 
probability of error obtained by the MAP detector are presented in Fig. [3] as a function of the distribution 
parameter, A 2 . The bounds depicted in this figure are: B^ BLBl \ B^ BLB2 \ B^ Bayesl ^> in addition to the 

proposed lower bounds B p with p = 1.5,2 and B p with p = 1.11,1.5,2. It can be seen that B^ BLB1 > is 

(2) 
lower than any proposed lower bound. In addition, for A 2 > 0.65 the proposed bound B\ ^ is tighter than all 

the other bounds and it is close to the minimum probability of error obtained by the MAP decision rule. For 

A 2 > 0.8 B\ I is tighter than the Bhattacharyya lower bounds and B\ and B2 are tighter than the B^ BLB1 > 
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B (2) 
1.1 



0.° 0.25 



. r( 2 ) , _+_ , p(Gauss-sin) 
1.25 b 



B< ATLB > a=5 




0.4 0.6 

POJx) 

Fig. 2 

The proposed lower bounds, B p and B v with p = 1.25, 1.1 compared to other existing bounds as a 

FUNCTION OF THE CONDITIONAL PROBABILITY P(#i|x) FOR BINARY HYPOTHESIS TESTING. 



everywhere and tighter than other bounds in some specific regions. Fig. [JJ presents the proposed lower bounds 
Bp with p = 1.5, 2 as a function of A2 compared to the upper bounds on the MAP probability of error [3], 
given by (f32j) . It can be seen that this class of upper bounds is proportional to the proposed tightest subclass 
of lower bounds in (1201) with a factor of 2 P_1 . 
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CD 

o- 0.25 




0.5 1 1.5 



2.5 



3.5 4 4.5 



K 



Fig. 3 

Comparison of the different lower bounds and the exact minimum probability of error as a 

function of a 2 for two equally-likely exponential distribution hypotheses. 



C. Example: Multiple hypothesis problem 



Consider the following multiple hypothesis testing problem: 



9 1 : f(x\6 1 ) = | cos 2 (x/2)e~\ x 
#2 ■■ f(x\9 2 ) = 2sin 2 (x/2)e-l x l 



(43) 



: f(x\9 3 



5^2( x ) e -|x| 



■ sin 



15 



0.6 



V © 

\ 

\ O 

\ : ' 



p(min) 

e 

-h-B( 1 > 

1.5 

"O " power mean upper bound, p=2 
- ▼ - power mean upper bound, p=1 .5 




0.5 1 1.5 



Fig. 4 

Comparison of the upper and lower bounds and the exact minimum probability of error as a 

function of a 2 for two equally-likely exponential distribution hypotheses. 



with P{0\) = J?, P{@2) = J?> an d P(0z) = Ss- In this problem, the exact probability of error of the MAP 
detector is difficult to compute. The bounds B( Ba v esl \ B^ Ba v es2 \ B^ Ba y es3 \ and B^ uad ^ are not tractable. 
The proposed bound with q = 2 is computable and is equal to 



B 



(i) 



40 



9=2 14 / 1 1 

-'° cos 2 (ie/2) ^ sin 2 (x/2) "*" sin 2 (a;) 



dx 



35 



e _x (cos(2z) - 2sin(2x) - 5) |g° = 0.2286 . 
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This example demonstrates the simplicity of the proposed bound with q = 2, while the other bounds are 
intractable. 

VI. Conclusion 

In this paper, new classes of lower bounds on the probability of error in multiple hypothesis testing were 
presented. The proposed classes depend on a parameter, p, which at the limit of p — >• 1 + approach the 
minimum attainable probability of error provided by the MAP detector. It is shown that these classes of 
bounds generalize some existing bounds for binary and multiple hypothesis testing. New variations using the 
proposed classes. It was shown via examples that the proposed bounds outperform other existing bounds in 
terms of tightness and simplicity of calculation. 

Appendix 

A. Necessary and sufficient condition for independency of flTJ) and flU) on 9 

In this appendix, it is shown that the expectation E [|u(x, 9)vf~ (x, 9)\] , k = 1,2 is independent of the 
detector 9 iff 



VfclX, 



Cfc(x) 



, fc = 1,2, i = l,...,M 



(44) 



P(0i\x) 

where Ci(') an d C2O) are arbitrary functions of the observations x. 

Sufficient condition: By substituting the function ffc(x, #,) = pf^^ for almost every x in E [|n(x, 9)vk (x, 0)\], 
one obtains 



P(9i\x) 



E[Kx,0K(x,0)|]=E 



M 



^n(x,^)C fc (x) 



i=l 



E 



M 



E c*w 



(M-l)E[C fc (x)] 



(45) 



which is independent of the detector 9. Substitution of ([43]) in (|7|) and © results in bounds that are inde- 
pendent of the detector 9. 

Necessary condition: Let E [|u(x, 6)vk (x, 9)\] be independent of the detector 9 and define the following 
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sequence of two-hypothesis detectors 



LJ.1 



(x) = 9jl x( zA + 8llx£A c 



9j if x e A 



, j,l = l,...,M 



(46) 



9i if x G A c 

where x is a random observation vector with positive probability measure and A c is the complementary event 
of A. For each detector 



E[Kx,0H(x,0)|] = 

M 
^P(^|x)w fc (x,6»i)l^ 9 . 



E 



i=l 



xei P(x e A) + E 

Using (|4"6"|) and the law of total probability, one obtains 



M 



^P(0i|x)u fe (x,0 i )l^ 



xei c 



P(x e A c ) . (47) 



E[Kx,% fc 


(M)|] = 




= E 


M 
^P(^|xK(x,^) 


x<E A 


= E 


£P(0i|x)t;*(x,0O 


x<E A 


+ E 


A/ 


x£ A 


= E 


1 M 

- Wi 


M 



P(x eA)+E 



P(x eA)-E 



P(x eA) + E 



a* 



^P(^|xK(x,^) 



1 = 1 

<\1 



^P(^|x)^(x,^) 



i=l 

L i^Z 



^P(^|x)^(x,^ 



1 = 1 



xe# 



xei 



xei c 



P(x G ,4 C 



P(x G A) + 



P(x G ^ c 



x G A 



P(x G A) + E 



M 



J2P(^|x)u fc (x,^ 



i=l 



E [ (P(0j|xH (x, e t ) - P{9 J \^)v k (x, 9 A) |x€i]P(xGi) + E 



JW 



^P(^|xK(x,^ 



i=l 



(48) 



Under the assumption that E [\u(x, 9)v k (x, 9)\] is independent of the detector 9, in particular, (|4T|) is iden- 
tical for all j, Z = 1, . . . , M, that is this term is independent of A, 9j, and 9\. Thus, for given 9i, the term 
E [(P(9i\x)vk (x, 9i) — P{9j\x)vk (x, 9j))\ x G A] P(x G ^4) is identical for every A and 0j. In particular, by 
setting A = where is the empty set, one obtains 



E [(P(0i|xH (x, 9i) - P(9 j \^)v k (x, 0,-))l x G 0] P(x G 0) = 0, j = 1, . . . ,M 
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and therefore 

E [P(9i\x)v k (x, e t ) - P(9j\x)v k (x, 6j) |x € A] P(x e A) = 0, VA, j = 1, . . . , M 
which is possible only if 

P(0j|x)i/ fc (x, 0,) - P(0j|x)t; fc (x, %) = 0, Vi = 1, . . . , M . 
Because I is arbitrarily chosen, one obtains 

P(0j|x)« fc (x,0f) = P(^-|x)t; fc (x,0,) = Cfe(x), Vj,/ = 1, . . . ,M 
where Cfc( x ) does not depend on the hypothesis. 

B. Review of Existing Lower Bounds 

In this appendix, some existing lower bounds on the minimum probability of error are presented. Part of 
these bounds are presented also in the review in }30j . 

Binary hypothesis testing bounds 
Most of the binary hypothesis testing bounds are based on divergence measures of the difference between two 
probability distributions, known as /-divergences or Ali-Silvey distances [31] , In [7], the divergence and two 
Bhattacharyya-based lower bounds were proposed. The divergence lower bound is 



p > B (<Mv) = I e -J/2 



(49) 



where J = E[logL(x)|#i] — E[logL(x)|#2] an d £(x) = P L |x\p/g 2 \ is the likelihood ratio function. A simple 
Bhattacharyya-based lower bound is 



E 2 

p > B (BLB1) = _ 



v / P(0i|x)P(^|x) 



(50) 



8P(9 1 )P(9 2 ) 

This bound is always tighter than the divergence lower bound [7]. The second Bhattacharyya-based bound 
on P P is 



Pe > B (BiB2) = l_lj 1 4E 2|^ (51) 
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Another /-divergence bound is proposed in [8] : 



1 1 



P e >B^ = ^-^/l -E[(4P(0i|x)P(0 2 |x))*] 



(52) 



where L > 1. For L = 1 this bound can be obtained also by applying Jensen's inequality on the MAP 
probability of error. The harmonic lower bound was proposed in [4]: 



P e > £( ffLB ) = E [P(0i|x)P(0 2 |x)] . 



(53) 



The pairwise separability measure, J a (9\x) = E [\P (#i|x) — P (#2|x)| a ], is used to derive the following 
binary bound [32] 



P e >P (Ja) = ^-^jJ(#|x), 1>«. 



(54) 



The "Gaussian-Sinusoidal" lower bound [TTJ is given by 



P e > ^( Gauss - Sin ) = 0.395E [sin(vrP(0 1 jx))e- Q ( p ( 9l l x )-^ 



(55) 



where a = 1.8063. Although this bound is tight, it is usually not tractable. An arbitrarily tight lower bound 
[TO] is given by 



P e > B^ ATLB ^ = -E 
a 



log 



1 + e" 



e -aP(0i|x) _|_ e -aP(6» 2 |x) 



(56) 



for any a > 0. By selecting high enough values for a, this lower bound can be made arbitrarily close to 
P e m . However, in general this bound is difficult to evaluate. 

Multiple hypothesis testing bounds 
For multiple hypothesis testing problems, the following lower bounds have been proposed. In [17] . Devijver 
derived the following bounds using the conditional Bayesian distance: 



(57) 



and 



, ^ u (Bayesl) _M-\ 


( 


M x E 


" M 

E^ 2 (^|x) 
.i=i 


-1 


e "^ M 


Af-1 




/ 


P e > B (Bayes2) = 1 - A 


E 

\ 


" M 


P 2 (^i|x) 







(58) 
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where E 



M 

i=l 



is the conditional Bayesian distance. The bound in ([57]) with M = 2 is identical to 
(|54|) with a = 2. In [17j . it is analytically shown that for the binary case the Bayesian distance lower bound in 
(|57p is always tighter than the Bhattacharyya bound in (|5ip . Using Jensen's inequality, the following bound 
is tighter than the bound in ([58]) [3J, [T7] 



R > B 1 - 3 ^ 68 ^ = 1 - E 



Al 



^P 2 (^|x) 

\ i=l 



(59) 



The bound 



p > B {quad) = }__^ E 

2 2 



Al 



E p2 (w 



*=i 



(60) 



was proposed in [23] and [21] in the context of Vajdas quadratic entropy and the quadratic mutual information, 
respectively. Note that the bound B^ quad > can be interpreted as an M-ary extension to the harmonic mean 
bound, presented in (j55]> . In [17], it is claimed that B^ uad ) < S^ " 8 * 2 ) < B( Ba w Ml ) . The affinity measure of 
information relevant to the discrimination among the M hypothesis is defined as lower bound on P e [19] 



p > p(MLB) M-l 



Al 



]P*(6i\x) 



.4 = 1 



M 



(61) 
20], [2JJ. Many 



The "general mean distance" between the M hypotheses is G a ^ = E ( E«=i P ($il x ) 
lower bounds on P e and upper bounds on pV nm > can be obtained from this distance [21]. For example, the 
binary bound in ([32]) and the following classes of bounds: 



P e > B {GMD ^ = 1 - G"% < a, l</3, - <f3 



T a,j3 



a 



Pe > B { - GMD2 ^ =1-G a0 , < Q, 1< p < - 

a 



(62) 



(63) 



It can be seen that by substituting j3 = 2, a = 1 in ([62]) we obtained the lower bounds in ([58]) . By substituting 
< a < 1 and — = j3 in (|62p or in ()63|) . one obtains the lower bound 



P« > 1 - E 



JU 



E^w*) 



a=l 



(64) 
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